<html>
<head><meta charset="utf-8"><title>Multi-stage caching  to save time · general · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/index.html">general</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Multi-stage.20caching.20.20to.20save.20time.html">Multi-stage caching  to save time</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="203599672"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Multi-stage%20caching%20%20to%20save%20time/near/203599672" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> drcarrybit <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Multi-stage.20caching.20.20to.20save.20time.html#203599672">(Jul 11 2020 at 14:51)</a>:</h4>
<p>From a very lofty perspective...  I wonder if there is a way to improve compile times with multi-step caching.</p>
<p>For myself, I don't care if it takes 5 hours for the rustc compiler to compile the first time...  What's far more important to me is the time it takes to recompile after making minor changes, for example the initial value of a single literal, or the internals of a single function.</p>
<p>What would happen if we cache the required data from each stage of the compiler, along with the information we would need to find and replace any changes in the output. Each stage could identify and recompile just the bits that changed.</p>
<p>That might not make sense, so let me try an example...   </p>
<p>Imagine we have a big application that we are recompiling after just changing the implementation of a single function without changing the function's definition.</p>
<p>What would it take to recompile and insert just that function's implementation at each stage? </p>
<p>The next stage would then be able to do a comparison of what's changed, and again just recompile and replace that bits that changed.</p>
<p>If at any time, the changes become overwhelming, invalidate/delete the rest of the cache, and proceed as normal.  </p>
<p>Even if we can only get this to work for 2 or 3 stages, it would seem that this could vastly improve the average compile time. </p>
<p>It seems like something that could be grown slowly, for example, first allowing only function internals to change in just the first couple of stages. Eventually, it may be possible to extend the technique to handle more.</p>
<p>Disk Space:<br>
I've seen that some people are reluctant to see the compiler waste valuable disk space. I think it's a very worthwhile exchange if it can save me time, especially if the cache is stored in a specially marked, easy to recognize location (folder named "cache"?) where the user would easily be able to delete the folder and save space when they are finished.</p>



<a name="203599845"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Multi-stage%20caching%20%20to%20save%20time/near/203599845" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Multi-stage.20caching.20.20to.20save.20time.html#203599845">(Jul 11 2020 at 14:55)</a>:</h4>
<p>Sounds like incremental compilation? Doesn't really work across stages but you almost never need more than one anyways. Just set <code>incremental = true</code> in config.toml.</p>



<a name="203601051"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Multi-stage%20caching%20%20to%20save%20time/near/203601051" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Multi-stage.20caching.20.20to.20save.20time.html#203601051">(Jul 11 2020 at 15:25)</a>:</h4>
<p>I don't think they are referring to bootstrap stages, but instead "compiler phases".  <span class="user-mention" data-user-id="320218">@drcarrybit</span> rustc has a caching system that works at a fine granularity.  You can read more about it here <a href="https://rustc-dev-guide.rust-lang.org/queries/incremental-compilation.html">https://rustc-dev-guide.rust-lang.org/queries/incremental-compilation.html</a> (and the rest of that chapter).   There is ongoing work to continue to improve it.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>